Papers with reasoning scenarios
COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pretrained language models have shown promising results on commonsense reasoning benchmark datasets. |
| Approach: | They propose a commonsense reasoning benchmark dataset with 4k sentence pairs . they propose 'gamified' model-in-the-loop setup to incentivize challenging samples . |
| Outcome: | The proposed benchmarks show that the proposed model achieves 71% standard accuracy and 51% pairwise accuracy, well below human performance. |
Explain-Analyze-Generate: A Sequential Multi-Agent Collaboration Method for Complex Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Multiagent debate (MAD) is a popular approach for large language models . however, the performance of LLMs is suboptimal in complex reasoning scenarios . |
| Approach: | They propose a sequential collaboration framework to enable agents to provide constructive assistance to peers by decomposing complex tasks into essential subtasks. |
| Outcome: | The proposed framework achieves the highest performance on 19 out of 23 tasks and lower costs compared to MAD. |
ARise: Towards Knowledge-Augmented Reasoning via Risk-Adaptive Search (2025.acl-long)
Copied to clipboard
Yize Zhang, Tianshu Wang, Sirui Chen, Kun Wang, Xingyu Zeng, Hongyu Lin, Xianpei Han, Le Sun, Chaochao Lu
| Challenge: | Large language models (LLMs) have impressive capabilities but their application in open-ended, knowledge-intensive, complex reasoning scenarios is limited. |
| Approach: | They propose a framework that integrates risk assessment of intermediate reasoning states with dynamic retrieval-augmented generation within a Monte Carlo tree search paradigm. |
| Outcome: | The proposed framework outperforms the state-of-the-art KAR methods by up to 23.10% and the latest RAG-equipped large reasoning models by upto 25.37%. |
ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Charts provide visual representations of data and are used for analyzing information, addressing queries, and conveying insights to others. |
| Approach: | They propose a chart-specific vision-language Instruction-following dataset with 191K instructions and a pipeline model that extracts chart data tables and inputs them into a LLM. |
| Outcome: | The proposed model can solve a wide range of chart-related tasks, achieving state-of-the-art results on four tasks. |
II-MMR: Identifying and Improving Multi-modal Multi-hop Reasoning in Visual Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on assessing the model’s overall accuracy without evaluating it on different reasoning cases. |
| Approach: | They propose a novel idea to identify and improve multi-modal multi-hop reasoning in VQA by using two new language prompts to find a reasoning path to reach its answer. |
| Outcome: | The proposed model improves multi-modal multi-hop reasoning in visual question answering (VQA) it finds that the proposed model is easy to answer, simply demanding “single-hop” reasoning, whereas only a few questions require “multi-hop.” |
START: Self-taught Reasoner with Tools (2025.emnlp-main)
Copied to clipboard
Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Bowen Yu, Binyuan Hui, Junyang Lin, Xiang Wang, Dayiheng Liu
| Challenge: | Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in complex reasoning through long chain-of-thought, yet they struggle with precise computations and algorithmic operations. |
| Approach: | They propose a training-free approach that activates LRMs’ latent tool-use capabilities through artificial hints and a framework that enables models to learn effective tool utilization through diverse hint patterns and rejection-based data synthesis. |
| Outcome: | Experiments show that START significantly improves state-of-the-art LRMs across challenging benchmarks, including competition-level mathematics (AMC23: 95.0%, AIME24: 75.6%) and graduate-level science questions (GPQA: 64.6%). |
CRAFTQA: A Code-Driven Adaptive Framework for Complex Structured Data Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing unified structured data question answering methods rely on a set of predefined functions, which restricts their ability to perform complex reasoning beyond these predefined operations. |
| Approach: | They propose a novel adaptive code-driven framework that generates code-based reasoning operations based on a question. |
| Outcome: | The proposed framework improves on multiple structured datasets on real-world scenarios. |